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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
7/12/2007 has been entered. 

Response to Amendment 

2. The amendment filed on 7/12/2007 has been entered. The amendments to 
claims 13, 16, and 23 overcomes the 35 USC 112 second paragraph issues raised in 
the Final Rejection mailed on 3/12/2007. 

Response to Arguments 

3. Applicant's arguments at pages 10-12 filed 7/12/2007 concerning the 35 USC 
102 rejections based applicants admitted prior art have been fully considered and are 
deemed to be persuasive since the bulls eye discussed at page 35 appears to be part of 
the fluoro-guided system. 

4. Applicant's arguments filed 7/12/2007 concerning the 35 USC 102 rejections 
based Kienzle have been fully considered, but, they are deemed not to be persuasive 
because Kienzle may track the real instrument or patient, however, this is done so the 
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real instrument and virtual graphical guides may be properly displayed, see paragraphs 
53- 55, while still requiring the surgeon to properly align the real instrument with the aid 
of the real and virtual features displayed to the surgeon. 



Claim Rejections - 35 USC §112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 13 and 16-22, 42, and 43 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

Claim 13; 

At lines 5, 6, 12, and 24 "its" and "it" is claimed however this claim limitation does 
not clearly refer a previous claim limitation. 
Claim 16: 

At line 29 "if is claimed however this claim limitation does not clearly refer a 
previous claim limitation. Dependent claims 17-22, 42, and 43 do not correct this issue. 
Claim 19: 

This claim claims "makes the augmented reality line of sight coincide with the 
path by moving into a position where said target point and said at least one graphical 
axis marker line up" but does not claim what is being moved into a position to make the 
line of sight coincide with the path. Thus, the metes and bounds of the claim is unclear. 
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A proper prior art analysis of the claims cannot be made because the metes and 
bounds of the claims are not definite and because the specification does clarify this 
claims. Thus, a prior art rejection or an indication of allowability cannot be made with 
the currently pending claims. In re Steele, 305 F.2d 859,134 USPQ 292 (CCPA 1962) 
(it is improper to rely on speculative assumptions regarding the meaning of a claim and 
then base a rejection under 35 U.S.C. 103 on these assumptions). 



Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



8. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 



9. Claims 13, 16-18, 22, 23, 31-35, 37, and 40-42 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Kienzle, III. US PGPub application no. 2002/ 
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0077540, in view of Frank Sauer, et al., Augmented Workspace: designing an AR 
testbed, Proceedings and ACM International Symposium on Augmented Reality, 2000, 
October 5-6, 2000, pages 47-53, cited on the PTO-892 mailed on 3/26/2004 is 
referenced in this application in the Background at page 3 and in the Detailed 
Description in the paragraph spanning pages 17 and 18. 
Claim 13: 

Kienzle teaches a method for augmented reality guided instrument positioning, 
comprising the steps of: 

rendering at least one graphics path guide for indicating a path for a real 
instrument to follow to a target, the graphics path guide being constructed in a way that 
it frames the path so as not to obstruct a view of a central part of the real instruments if 
the real instruments along its axis in correct alignment to said path (Paragraphs 42-46 
describes and figures 7 and 8 show virtual probe tip portion 156 as a dashed outline 
which frames the path the instrument should follow to reach the target159. Paragraph 
44 discusses alternative graphic representations of the probe.); 

Kienzle does not fully teach displaying the rendered at least one graphics path 
guide on a display overlaid onto a direct optical or video view that contains said real 
instrument and a real object, which includes said target (However, the Augmented 
Workspace article teaches in section 6 a video see through HUD and in sections 9 and 
10 superimposing graphic guides onto the video of the real instrument and the real 
object. The article also discusses optical see through system in section 3.); 
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The combination of Kienzle and the Augmented Workspace article further 
teaches: 

a user performing the following steps: 

moving said real instrument, observed in said video view, to align it with the at 
least one graphic path guide, observed on the display that show the at least one graphic 
guide overlaid onto said view of said real instrument and said real object (Kienzle: 
paragraphs 39-46 Article: sections 9 and 10 and figure 7.) 

aligning the real instrument with the path by determining when the at least one 
graphics path guide frames the path so that a view of a central part of the real 
instrument is not obstructed by the at least one graphics path guide (Kienzle: 
paragraphs 39-46 Article: sections 9 and 10 and figure 7.); and 

moving said real instrument along the path so that a front portion of said 
real instrument is inserted into the object until its tip reaches said target (Kienzle: 
paragraphs 39-46 Article: sections 9 and 10 and figure 7.)- 

It would have been obvious to one of ordinary skill in the art to display the 
rendered at least one graphics path guide on Kienzle^s display overlaid onto a direct 
optical or video view that contains said real instrument and a real object in view of the 
Augmented Workspace article because the article teaches in sections 1 and 10 that 
augmented reality with video of the real scene provides beneficial results. 
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Claim 16: 

Kienzle teaches a method for augmented reality guided instrument positioning, 
comprising the steps of: 

defining a target point within a real object (figures 7 and 8, target 159); 

defining a path to reach the target point with a real instrument (Paragraphs 42-46 
describes and figures 7 and 8 show virtual probe tip portion 156 as a dashed outline 
which frames the path the instrument should follow to reach the target 159. Paragraph 
44 discusses alternative graphic representations of the probe.); 

Kienzle does not fully teach rendering a graphical representation of the path in 
the form of at least one graphical axis marker (Kienzle: 156) on a display overlaid onto 
a direct optical or video view of a real scene that comprises said real instrument, said 
real object, and includes said target point, so that a user can perceive said real 
instrument and said at least one graphical axis marker and their spatial relationship in 
an augmented reality view (However, the Augmented Workspace article teaches in 
section 6 a video see through HUD and in sections 9 and 10 superimposing graphic 
guides onto the video of the real instrument and the real object.), 

The combination of Kienzle and the Augmented Workspace article further 
teaches: 

a user performing the following steps: 

aligning said user's augments reality line-of-sight with the at least one graphical 
axis marker so that said path to reach said target point with the real instrument is 
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aligned along said augmented reality line-of-sight (Kienzle: paragraphs 39-46 Article: 
sections 9 and 10 and figure 7.) 

aligning the real instrument to the path by aligning the real instrument with the 
augmented reality line-of-sight towards the target point (Kienzle: paragraphs 39-46 
Article: sections 9 and 10 and figure 7.); and 

moving the real instrument along the path towards the target point while keeping 
it aligned with the augmented reality line-of-sight (Kienzle: paragraphs 39-46 Article: 
sections 9 and 10 and figure 7.). 

It would have been obvious to one of ordinary skill in the art to display the 
rendered at least one graphics path guide on Kienzle's display overlaid onto a direct 
optical or video view that contains said real instrument and a real object in view of the 
Augmented Workspace article because the article teaches in sections 1 and 10 that 
augmented reality with video of the real scene provides beneficial results. 

Claim 17: 

Both Kienzle and the article teach a real instrument that has an axis marker by 
being a needle. 
Claim 18: 

Both Kienzle and the article teach a real instrument that has an axis marker by 
being a needle. Thus, it is added. 
Claim 22: 

Kienzel teaches the method according to claim 16, wherein the at least one 
identified graphical axis marker comprises at least two axis markers for controlling 
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alignment of the real instrument along a line of sight (see figures 7 and 8, virtual probe 
tip portion 156 is a dashed outline having two axis markers). 
Claim 42: 

Kienzel teaches the method of claim 16 wherein the graphical target point and 
the graphical axis marker are designed such that information corresponding to the 
distance between the actual instrument and the point on the actual target can be directly 
observed from an alignment of the graphical target point and the graphical axis marker 
by graphically displaying the instrument as it is moved by the user relative to the target 
which shows the distance graphically. 

Claim 23: 

Kienzle teaches a method for augmented reality guided instrument positioning, 
comprising the steps of: 

defining a real target point within a real object (figures 7 and 8, target 159); 

defining a path to reach the target point within said real object (Paragraphs 42- 
46 describes and figures 7 and 8 show virtual probe tip portion 156 as a dashed outline 
which frames the path the instrument should follow to reach the target 159. Paragraph 
44 discusses alternative graphic representations of the probe. The dashed outline 
defines a path to reach the real target within the patient.); 

tracking a pose of a real instrument with respect to a pose of the real object 
(Paragraphs 42-46 and 53- 55 ); 

Kienzle does not fully teach rendering a graphical representation of the real 
instrument and the path to obtain a virtual instrument and a graphical virtual guide on a 
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display, the graphical representation being rendered with respect to a virtual viewpoint 
from which a virtual line of sight coincides with a virtual path for the virtual instrument to 
follow during a positioning of the real insure to the real target point, the graphical virtual 
guide correspond to the path, the virtual instrument comprising a 3D structure for line of 
sight alignment, the 3D structure comprising a plurality of markers centered on a 
distributed along an axis of the virtual instrument (However, the Augmented Workspace 
article teaches in section 6 a video see through HUD and in sections 9 and 10 
superimposing graphic guides onto the video of the real instrument and the real object. 
The article also discusses optical see through system in section 3.); 

The combination of Kienzle and the Augmented Workspace article further 
teaches: 

aligning the virtual instrument along the virtual fine of sight according to 
the graphical virtual guide in order to accordingly align the real instrument along the 
path (Kienzle: paragraphs 39-46 Article: sections 9 and 10 and figure 7.); and 

moving the real instrument by a user in response to viewing the virtual instrument 
and said graphical virtual guide on the display, the real instrument moving along the 
path towards the real target point keeping the correct alignment by observing and 
keeping in the alignment with the virtual instrument and graphical virtual guide (Kienzle: 
paragraphs 39-46 Article: sections 9 and 10 and figure 7.) 

It would have been obvious to one of ordinary skill in the art to display the 
rendered at least one graphics path guide on Kienzle's display overlaid onto a direct 
optical or video view that contains said real instrument and a real object in view of the 
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Augmented Workspace article because the article teaches in sections 1 and 10 that 
augmented reality with video of the real scene provides beneficial results. 
Claim 31: 

Kienzel teaches the method of claim 23, wherein the step of aligning the virtual 
instrument further comprises the step of choosing an orientation of the graphical 
representation around the virtual line of sight according to a pose of a user with respect 
to the real target point {by sensing the orientation of the real instrument the system has 
sensed the pose of the user). 

Claim 32: 

Kienzel teaches the method of claim 31, further comprising the step of 
determining the orientation such that east, west, north, and south correspond to right, 
left, fonA/ard, and backward, respectively, for the pose of the user in which the user 
faces the real target point, said determining step based on a selection {by sensing the 
orientation of the real instrument the system has sensed the pose of the user). 

Claim 33: 

Kienzel teaches the method of claim 31, wherein the orientation is dynamically 
adjusted according to a change of the pose of the user {by sensing the orientation of the 
real instrument the system has sensed the pose of the user). 

Claim 34: 

Kienzel teaches the method of claim 32, wherein the selection is dynamically 
adjusted with respect to the pose of the user {by sensing the orientation of the real 
instrument the system has sensed the pose of the user). 
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Claim 35: 

Kienzel teaches the method of claim 23 wherein the rendering step further 
comprises the step of rendering graphical information about a distance between the 
actual instrument and the point on the actual target, the graphical information about the 
distance being overlaid onto the graphical representation by graphically displaying the 
instrument as it is moved by the user relative to the target which shows the distance 
graphically. 

Claim 37: 

Kienzel teaches the method of claim 23 wherein the virtual target point and the 
virtual instrument are designed such that information corresponding to the distance 
between the actual instrument and the point on the actual target can be directly 
observed from an alignment of the virtual target point and the virtual instrument by 
graphically displaying the instrument as it is moved by the user relative to the target 
which shows the distance graphically. 

Claim 40: 

Kienzel teaches the method of claim 23, wherein the graphical representation 
from the virtual viewpoint is combined with another graphical representation from 
another virtual viewpoint looking at the virtual path from a side thereof with reference to 
figures 4-7, 11, and 12. 
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Claim 41: 

Kienzel teaches the method of claim 23, wherein said graphical representation 
from the virtual viewpoint is combined with an augmented reality view with reference to 
figures 4-7, 11, and 12. 

10. Claims 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kienzle, III. US PGPub application no. 2002/ 0077540, in view of Frank Sauer, et al.. 
Augmented Workspace: designing an AR testbed, Proceedings and ACM International 
Symposium on Augmented Reality, 2000, October 5-6, 2000, pages 47-53, cited on the 
PTO-892 mailed on 3/26/2004 is referenced in this application in the Background at 
page 3 and in the Detailed Description in the paragraph spanning pages 17 and 18 and 
in view of applicants admission of the prior art and motivation of using aiming aids of 
guns with instrument guidance found at page 38 lines 7-16. 

Kienzel does not teach the graphical axis marker comprises an intersection of at 
least two lines, the intersection to be centered on the axis of the actual instrument for 
correct alignment. 

It would have been obvious to one of ordinary skill in the art to use the feature of 
gun aiming, the cross, and apply it to augmented reality aiming of real instruments 
because this will give the user a good idea of the alignment of the real instruments with 
the real target. 
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11. Claim 38 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kienzle, 
III. US PGPub application no. 2002/ 0077540, in view of Franl< Sauer, et al.. 
Augmented Worl<space: designing an AR testbed. Proceedings and ACIVI International 
Symposium on Augmented Reality, 2000, October 5-6, 2000, pages 47-53, cited on the 
PTO-892 mailed on 3/26/2004 is referenced in this application in the Background at 
page 3 and in the Detailed Description in the paragraph spanning pages 17 and 18. 

Kienzel does not teach rendering by using a virtual camera with a wide angle 

lens. 

Page 50 line 24 to page 51 line 4 of applicants specification does not give a 
reason for rendering by using a virtual camera with a wide angle lens. 

It would have been obvious to one of ordinary skill In the art to render by using a 
virtual camera with a wide angle lens because this is one of many ways to render the 
graphical images and because applicant did not give any reason for choosing this type 
of rendering. 

12. Claims 20, 25, 26, 29, 30, 39, and 43 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Kienzle, III. US PGPub application no. 2002/ 0077540, in view 
of Frank Sauer, et al.. Augmented Workspace: designing an AR testbed. Proceedings 
and ACM International Symposium on Augmented Reality, 2000, October 5-6, 2000, 
pages 47-53, cited on the PTO-892 mailed on 3/26/2004 is referenced in this application 
in the Background at page 3 and in the Detailed Description in the paragraph spanning 
pages 17 and 18 and in view of applicants admission of the prior art and motivation of 
using aiming aids of guns with instrument guidance found at page 38 lines 7-16 and in 
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view of the prior art bulls eye discussed at page 35 lines 12-20 and page 35 line 26 to 
page 36 line 2. 

A bulls eye has concentric circular shapes having different diameters about and 
centered on an axis and this configuration together with the real instrument achieves a 
pre-defined configuration when the real instrument reaches the real target point. 

Each of these claims have varying degrees of limitations but all these claims are 
rejected based upon the bulls eye feature and based upon aiming aids of guns. 

It would have been obvious to one of ordinary skill in the art to use a bulls eye 
type of axis marker as a graphical aid in the augmented and virtual environment of 
Kienzle as modified by the Augmented Workspace article because circles and 
concentric circles provide a art recognized means of allowing a user to align work 
pieces, fire arms, etc. to a target point. 

13. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Tuceryan, M.; Greer, D.S.; Whitaker, R.T.; Breen, D.E.; Crampton, C; Rose, E.; 
Ahlers, K.H.; Calibration Requirements and Procedures for a Monitor-Based Augmented 
Reality System, September 1995, vol. 1, no. 3, pages 255-273, is referenced in this 
application in the Background at page 3 and in the Detailed Description in the 
paragraph spanning pages 17 and 18. 

Tuceryan, M.; Navab, N.; Single point active alignment method (SPAAM) for 
optical see-through HMD calibration for AR, 5-6 Oct. 2000, pages 149-158, is 
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referenced in this application in the Background at page 3 and in the Detailed 
Description in the paragraph spanning pages 17 and 18. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffery A Brier whose telephone number is (571) 272- 
7656. The examiner can normally be reached on M-F from 7:30 to 4:00. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, Michael 
Razavi, can be reached at (571) 272-7664. The fax phone Number for the organization 
where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



/Jeffery A. Brier/ 

Primary Examiner, Division 2628 



